Systems and methods for model-based image analysis

ABSTRACT

A system for categorizing images is provided. The system is programmed to store a first training set of images. Each image of the first training set of images is associated with an image category of a plurality of image categories. The system is further programmed to analyze each image of the first training set of images to determine one or more features associated with each of the plurality of image categories and receive a second training set of images. The second training set of images includes one or more errors. The system is also programmed to analyze each image of the second training set of images to determine one or more features associated with an error category and generate a model to identify each of the image categories based on the analysis such that the model includes the error category in the plurality of image categories.

BACKGROUND

The field of the present disclosure relates generally to analyzingimages and, more specifically, to analyzing images to categorize anddetermine issues with those images.

Evaluation of images is labor-intensive process and may be dependent onsubject matter expertise. The more complicated the image and the morepotentially expensive that having errors on an image are, the morecareful each image needs to be carefully analyzed. Trained individualsmay have to spend significant time analyzing images for potential flawsand/or errors. For systems where multiple images need to be analyzed,the process may become extremely time-intensive. In addition, some knownimage evaluation systems rely on rudimentary automated checks, which areonly able to identify a few basic error conditions. Accordingly, it isadvisable to determine additional systems for analyzing images forerrors.

BRIEF DESCRIPTION

In one aspect, a system for categorizing images is provided. The systemincludes a computing device having at least one processor incommunication with at least one memory device. The at least oneprocessor is programmed to store a first training set of images. Eachimage of the first training set of images is associated with an imagecategory of a plurality of image categories. The at least one processoris also programmed to analyze each image of the first training set ofimages to determine one or more features associated with each of theplurality of image categories. The at least one processor is furtherprogrammed to receive a second training set of images. The secondtraining set of images includes one or more errors. In addition, the atleast one processor is programmed to analyze each image of the secondtraining set of images to determine one or more features associated withan error category. Moreover, the at least one processor is programmed togenerate a model to identify each of the image categories based on theanalysis such that the model includes the error category in theplurality of image categories.

In another aspect, a method for categorizing images is provided. Themethod is implemented by a computer device having at least one processorin communication with at least one memory device. The method includesstoring a first training set of images. Each image of the first trainingset of images is associated with an image category of a plurality ofimage categories. The method also includes analyzing each image of thefirst training set of images to determine one or more featuresassociated with each of the plurality of image categories. The methodfurther includes receiving a second training set of images. The secondtraining set of images includes one or more errors. In addition, themethod includes analyzing each image of the second training set ofimages to determine one or more features associated with an errorcategory. Moreover, the method includes generating a model to identifyeach of the image categories based on the analysis such that the modelincludes the error category in the plurality of image categories.

In a further aspect, a computer device for categorizing images isprovided. The computer device includes at least one processor incommunication with at least one memory device. The at least oneprocessor is programmed to store a first training set of images. Eachimage of the first training set of images is associated with an imagecategory of a plurality of image categories. The at least one processoris also programmed to analyze each image of the first training set ofimages to determine one or more features associated with each of theplurality of image categories. The at least one processor is furtherprogrammed to receive a second training set of images. The secondtraining set of images includes one or more errors. In addition, the atleast one processor is programmed to analyze each image of the secondtraining set of images to determine one or more features associated withan error category. Moreover, the at least one processor is programmed togenerate a model to identify each of the image categories based on theanalysis such that the model includes the error category in theplurality of image categories. Furthermore, the at least one processoris programmed to receive an image. The image includes a metadata imagecategory. In addition, the at least one processor is also programmed toexecute the model to analyze the received image. In addition, the atleast one processor is further programmed to identify an image categoryof the plurality of image categories for the image based on theexecution of the model. Moreover, the at least one processor is alsoprogrammed to compare the identified image category to the metadataimage category to determine if there is a match. If there is a match,the at least one processor is programmed to approve the image. If thereis not a match, at least one processor is programmed to flag the imagefor further review.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a block diagram of a system for training an imagecategorization and learning (ICL) model in accordance with oneembodiment of the present disclosure.

FIG. 2A illustrates an exemplary clean image that may be used to trainthe ICL model shown in FIG. 1.

FIG. 2B illustrates an exemplary image with errors that may be detectedby the ICL model shown in FIG. 1.

FIG. 3 illustrates a block diagram of a system for using a trained imagecategorization and learning (ICL) model that has been trained using thesystem 100 shown in FIG. 1.

FIG. 4 is a simplified block diagram of an example system for executingthe systems shown in FIGS. 1 and 3.

FIG. 5 illustrates an example configuration of a client computer deviceshown in FIG. 4, in accordance with one embodiment of the presentdisclosure.

FIG. 6 illustrates an example configuration of the server system shownin FIG. 4, in accordance with one embodiment of the present disclosure.

DETAILED DESCRIPTION

The implementations described herein relate to systems and methods foranalyzing images and, more specifically, to analyzing images tocategorize and determine issues with those images. In particular, animage categorization and learning (ICL) model is executed by a computingdevice to (1) learn to identify the categories associated with aplurality of types of images; (2) learn to identify errors in theplurality of types of images; and (3) identify a category for an imageand one or more errors to that image based on the identified class.

In the exemplary embodiment, the ICL model is first trained using alabeled training set of images, where the images have labels thatidentify the category of image. The ICL uses the features of thoseimages to determine how to recognize different types of images. When theICL model is trained to categorize images, the ICL model is then used toreceive images and identify the image category based on the imageitself. Next, the ICL model is trained to identify issues and/or errorswith the images. In this phase, the ICL model receives images witherrors and is trained to put those images in an error category.

During normal use of the ICL model, the model receives an image with aknown image category. For example, the image category may be stored inthe metadata of the image. The ICL model determines the image categorybased on the image and compares the determined image category to the onestored in the metadata. If there is a match, then the system moves on tothe next image. If there is a discrepancy, the system submits the imageand determined image category to a user for review. If the image has anerror, then the ICL model puts the image in the error image category. Inat least one embodiment, the ICL model is trained to identify an imagecategory for an image and one or more errors in the image based on thatimage category.

Described herein are computer systems such as the ICL computer devicesand related computer systems. As described herein, all such computersystems include a processor and a memory. However, any processor in acomputer device referred to herein may also refer to one or moreprocessors wherein the processor may be in one computing device or aplurality of computing devices acting in parallel. Additionally, anymemory in a computer device referred to herein may also refer to one ormore memories wherein the memories may be in one computing device or aplurality of computing devices acting in parallel.

As used herein, a processor may include any programmable systemincluding systems using micro-controllers, reduced instruction setcircuits (RISC), application-specific integrated circuits (ASICs), logiccircuits, and any other circuit or processor capable of executing thefunctions described herein. The above examples are example only, and arethus not intended to limit in any way the definition and/or meaning ofthe term “processor.”

As used herein, the term “database” may refer to either a body of data,a relational database management system (RDBMS), or to both. As usedherein, a database may include any collection of data includinghierarchical databases, relational databases, flat file databases,object-relational databases, object oriented databases, and any otherstructured collection of records or data that is stored in a computersystem. The above examples are example only, and thus are not intendedto limit in any way the definition and/or meaning of the term database.Examples of RDBMS' include, but are not limited to including, Oracle®Database, MySQL, IBM® DB2, Microsoft® SQL Server, Sybase®, andPostgreSQL. However, any database may be used that enables the systemsand methods described herein. (Oracle is a registered trademark ofOracle Corporation, Redwood Shores, Calif.; IBM is a registeredtrademark of International Business Machines Corporation, Armonk, N.Y.;Microsoft is a registered trademark of Microsoft Corporation, Redmond,Wash.; and Sybase is a registered trademark of Sybase, Dublin, Calif.)

In another embodiment, a computer program is provided, and the programis embodied on a computer-readable medium. In an example embodiment, thesystem is executed on a single computer system, without requiring aconnection to a server computer. In a further example embodiment, thesystem is being run in a Windows® environment (Windows is a registeredtrademark of Microsoft Corporation, Redmond, Wash.). In yet anotherembodiment, the system is run on a mainframe environment and a UNIX®server environment (UNIX is a registered trademark of X/Open CompanyLimited located in Reading, Berkshire, United Kingdom). In a furtherembodiment, the system is run on an iOS® environment (iOS is aregistered trademark of Cisco Systems, Inc. located in San Jose,Calif.). In yet a further embodiment, the system is run on a Mac OS®environment (Mac OS is a registered trademark of Apple Inc. located inCupertino, Calif.). In still yet a further embodiment, the system is runon Android® OS (Android is a registered trademark of Google, Inc. ofMountain View, Calif.). In another embodiment, the system is run onLinux® OS (Linux is a registered trademark of Linus Torvalds of Boston,Mass.). The application is flexible and designed to run in variousdifferent environments without compromising any major functionality.

In some embodiments, the system includes multiple components distributedamong a plurality of computer devices. One or more components may be inthe form of computer-executable instructions embodied in acomputer-readable medium. The systems and processes are not limited tothe specific embodiments described herein. In addition, components ofeach system and each process can be practiced independent and separatefrom other components and processes described herein. Each component andprocess can also be used in combination with other assembly packages andprocesses. The present embodiments may enhance the functionality andfunctioning of computers and/or computer systems.

As used herein, an element or step recited in the singular and proceededwith the word “a” or “an” should be understood as not excluding pluralelements or steps, unless such exclusion is explicitly recited.Furthermore, references to “example embodiment” or “one embodiment” ofthe present disclosure are not intended to be interpreted as excludingthe existence of additional embodiments that also incorporate therecited features.

As used herein, the terms “software” and “firmware” are interchangeable,and include any computer program stored in memory for execution by aprocessor, including RAM memory, ROM memory, EPROM memory, EEPROMmemory, and non-volatile RAM (NVRAM) memory. The above memory types areexample only, and are thus not limiting as to the types of memory usablefor storage of a computer program.

Furthermore, as used herein, the term “real-time” refers to at least oneof the time of occurrence of the associated events, the time ofmeasurement and collection of predetermined data, the time to processthe data, and the time of a system response to the events and theenvironment. In the embodiments described herein, these activities andevents occur substantially instantaneously.

The systems and processes are not limited to the specific embodimentsdescribed herein. In addition, components of each system and eachprocess can be practiced independent and separate from other componentsand processes described herein. Each component and process also can beused in combination with other assembly packages and processes.

The patent claims at the end of this document are not intended to beconstrued under 35 U.S.C. § 112(f) unless traditionalmeans-plus-function language is expressly recited, such as “means for”or “step for” language being expressly recited in the claim(s).

For the purposes of this discussion, terminal charts will be used as anexample of the type of images that may be categorized and analyzed.Terminal charts are maps for pilots to understand procedures for flyinginto or out of an airport. Categories for terminal charts may include,but are not limited to, approach, arrival, departure, and airport map.In the exemplary embodiment, terminal charts are vector-based graphicfiles. Terminal charts may be generated by hand or through the use ofautomated graphic generation programs, such as by conversion from rastergraphics, scanned images, or previous version from other programs.

FIG. 1 illustrates a block diagram of a system 100 for training an imagecategorization and learning (ICL) model 125 in accordance with oneembodiment of the present disclosure.

In the exemplary embodiment, a first model-training module 105 is usedfor the initial training of the ICL model 125. The ICL model 125 isfirst trained using an image library 110 containing a plurality ofimages and image metadata 115 associated with the plurality of images.In the exemplary embodiment, the plurality of images in the imagelibrary 110 may be any type of image that could be broken intocategories. This may include, but is not limited to, terminal charts foraircraft, topographical maps, medical images (i.e., x-rays,electrocardiograms, sonograms), and photographs. Images may includevector-based graphics or raster graphics. The image metadata 115 mayinclude, but is not limited to, name, effective date/date taken, imageformat, and image category. The image library 110 and the image metadata115 are combined to generate a first training set 120. In the exemplaryembodiment, the first training set 120 includes a large number of imagesfrom each category or type. For example, the first training set 120 mayinclude over 10,000 to over 100,000 images covering the different imagecategories.

In the first training set 120, each image is associated with an imagecategory. The first model training module 105 uses the first trainingset 120 to train the ICL model 125 in categorizing images into one ofthe image categories. In the exemplary embodiment, the first modeltraining module 105 knows the image category for each image in thetraining set, so the first model training module 105 is able to comparedifferent images of the same category and different images of differentcategories to determine which features in each image is indicative ofeach image category to train the ICL model 125. The ICL model 125determines what are common features of a particular category of image,such as an approach chart, which makes the approach chart distinct fromother types of images, such as a departure chart. These features mayinclude small subtleties of the image, such as in the placement of alogo or index number, the flow of lines, the sizes of numbers, thedirections of arrows, and/or any number of additional features.

In some embodiments, images of the same category may have some differentfeatures, for example, a type of terminal chart that has changed overtime (i.e. reformatted) or photographs from different types anddifferent eras of cameras.

After the first training of the ICL model 125, the ICL model 125 iscapable of receiving an image and determining which, if any, of theimage categories that the image should be labeled as. Examples of imagecategories include, but are not limited to, indoors image, outsideimage, type of terminal chart (approach, arrival, or departure),topographical map type, medical image type and/or orientation, artworktype (painting, drawing, print, decorative arts, or sculpture), artworktheme (portrait, landscape, religious, or abstract), artwork style(Modernism, Cubism, Surrealism, Expressionism, and Impressionism),categorization based on subject matter (i.e., mountains, people,sporting event), vector-based graphics, raster graphics, or any othercategorization as needed.

Next, the ICL model 125 is trained a second time by the secondmodel-training module 130, this time with images with errors. In theexemplary embodiment, a second training set 135 is provided. In theexemplary embodiment, the second training set 135 includes a pluralityof images with one or more errors. These errors may include, but are notlimited to, rendering errors (wherein the image is not properlyrendered), missing items or features in the image, duplicate items orfeatures in the image, items covered up by other items, blank images,all white or all black images, line rendering is too heavy or too light,misclassification of image category, misspellings, lower resolution thanexpected, blurred features or sections, and inverted colors. In someembodiments, the second training set 135 only contains images witherrors. In other embodiments, the second training set 135 includesimages with errors and images without errors.

For each image in the second training set 135, the ICL model 125receives the image and categorizes the image. In the exemplaryembodiment, the second model-training module 130 trains the ICL model torecognize a new category of image, an error category. In someembodiments, the error category is a simple image category thatrepresents that there is an error in the image. In other embodiments,the error category includes a plurality of sub-categories that indicatethe type of error. For example, some sub-categories may indicate a titleerror, a rendering error, or a misclassification error. Somesub-categories may be associated with different image categories. Forexample, the sub-category may indicate that the image is an arrivalchart with an error or even an arrival chart with a title error. Thesecond model-training module 130 determines 140 the error category foreach image with an error and trains the ICL model 125 to recognize thoseerror categories.

The second model-training module 130 updates the ICL model 125 into atrained ICL model 145. In some embodiments, the ICL model 125 is trainedto be updated into the trained ICL model 145. In other embodiments, thetrained ICL model 145 is generated from the ICL model 125 and the secondtraining set 135. In still further embodiments, the trained ICL model145 is generated from scratch using the information in the ICL model 125and the second training set 135.

In the exemplary embodiment, the second model training module 130 andsecond training set 135 are configured to train the ICL model 125 torecognize errors and generate an error category. This error category mayinclude images that are not categorized properly, not rendered properly,or otherwise do not fit into any of the categories that the ICL model125 was trained to recognize by the first model-training module 105. Insome embodiments, the ICL model 125 is further trained to identifydifferent error types and report those errors types when they arecontained in an image. In one example, the ICL model 145 may recognizean image as an arrival chart, but be able determine that one or moreerrors are contained in that chart, such as some of the text is fuzzy.This addition of an error category allows the ICL model 145 to not onlyidentify general errors through category mismatch, but to identifypotential errors through category discrepancies.

While the above describes only a first training set 120 and a secondtraining set 135, system 100 may include a plurality of training sets.The system 100 may use different training sets to recognize differenttypes of errors and error categories. For example, the system 100 mayinclude a training set just to train the ICL model 125 in regards tomisclassification errors. In addition, in some embodiments, the firsttraining set 120 and the second training set 135 may be combined into asingle training set and the first model-training module 105 and thesecond model-training module 130 may be combined into a singlemodel-training module.

In some further embodiments, the trained ICL model 145 may also betrained to recognize individual objects or features in an image aserrors. This may allow the trained ICL model 145 to categorize errorsinto sub-categories based on the individual errors.

Examples of error types include, but are not limited to, renderingerrors (where all or a portion of the image did not render, such asextra white space or black space), fuzzy text, fuzzy lines, missingfeatures, extraneous features, no known category, misclassification,incorrect orientation of image, incorrect orientation of features,incorrect text, incorrect lines, incorrect resolution, incorrect size,inverted color scheme, incorrect color scheme, and/or any other errorbased on the use case.

In some embodiments, the trained ICL model 145 is trained to recognizean image and categorize that image based on a portion of that image,rather than having to analyze the entire image. In these embodiments,the trained ICL model 145 is trained to recognize features of the imageand analyze whether or not those features are present and correct tocategorize the image.

FIG. 2A illustrates an exemplary clean image 200 that may be used totrain the ICL model 125 (shown in FIG. 1). FIG. 2B illustrates anexemplary image 250 with errors that may be detected by the ICL model125 (shown in FIG. 1). Example errors shown in FIG. 2A include titleerrors 205, where the title of the chart is listed twice and overlaps tocreate fuzzy lettering. The image 250 also includes two rendering errors210 and 215. In this case, rendering error 210 is a white line, wherethe image 250 did not completely render or skipped over information.Rendering error 215 is a black space where the end of the image 250 didnot finish rendering the image, such as through a corruption of theimage file. Typo Errors 220 and 225 show where numbers have been removedor added. In some embodiments, the ICL model 125 is capable of analyzingthe content of the image 250 and determining if there are errors. Suchas in an arrival chart where the numbers increase where they shoulddecrease or vice versa. The errors 205-225 shown in here are onlyexamples, and the ICL model 125 may be configured to detect and identifyother errors based on its training, such as those errors contained inthe second training set 135 or based on which features the ICL model 125uses to determine each classification.

FIG. 3 illustrates a block diagram of a system 300 for using a trainedimage categorization and learning (ICL) model 145 that has been trainedusing the system 100 (both shown in FIG. 1).

In the exemplary embodiment, image production 305 produces a set ofimages 310. In some embodiments, image production 305 is a conversionprogram that converts files from one file type to another file type. Inother embodiments, image production 305 is a communications program thatreceives the set of images 310 that have been transmitted from anothercomputer device. In still further embodiments, image production 305creates the set of images 310. In some of these embodiments, imageproduction 305 creates the image from one or more datasets, such as inthe situation where the set of images 310 are procedurally created. Inadditional embodiments, image production 305 includes one or more usersthat create or modify the set of images 310.

In the exemplary embodiment, a data validation module 315 is programmedto use the trained image categorization and learning (ICL) model 145 tocategorize and analyze the set of images 310. The data validation module315 loads 320 each image in the set of images 310. In some embodiments,the data validation module 315 loads 320 and validates each imageserially. In other embodiments, the data validation module 315 may load320 and validate multiple images in parallel based on availablecomputing resources.

The data validation module 315 applies 330 the trained ICL model 145 tothe image to categorize the image. In the exemplary embodiment, thetrained ICL model 145 analyzes the features of the image to determinewhich image category that the image should be categorized as. In theexemplary embodiment, the image includes metadata that includes theimage category. The determined image category is compared 330 to themetadata image category. If the determined image category matches themetadata image category, then the data validation module 315 reportssuccess 335 and continues to the next image. In some embodiments, thetrained ICL model 145 may classify the image in the same image categoryas the metadata image category, but also list that the image has anerror. In some other embodiments, the trained ICL model 145 may onlyclassify the image as being in an error image category. In either ofthese cases, the image is flagged for further review 340.

In the exemplary embodiment, there are three types of errors that thetrained ICL model 145 may identify. First, the trained IDL model 145will identify when there is a discrepancy between the predicted imagecategory (as determined by the trained ICL model 145) and the actualimage category (such as in the metadata). Next, the trained IDL model145 will identify when the image does not fit into any image category(such as the wrong file being in the wrong place). And finally, thetrained IDL model 145 will identify when an image matches a known errorcategory.

In some embodiments, the further review 340 is performed by one or moreusers. In other embodiments, the further review 340 is performed byanother module, model, or computer device. The further review 340determines 345 if a valid image issue has been identified. For example,if the further review 340 determines 345 that one or more of the errors205-225 (shown in FIG. 2B) appear in the image, then the data validationmodule 315 would mark 350 that image as needing correction. In someembodiments, the data validation module 315 would transmit the markedimage back to the image production 305 for correction. If the furtherreview 340 determines that the image is valid, then the data validationmodule 315 may transmit the image and metadata to update the trained ICLmodel 145.

In some further embodiments, ICL model 125 (shown in FIG. 1) may betrained by system 300 to be updated into trained ICL model 145.

For example, the system 300 may be used to validate terminal chartimages. In one example, a computer program may convert terminal chartsfrom one format to another format. In this example, the system 300 maybe used to validate each of the terminal charts after they have beenconverted. The system 300 may then detect when there are problems withany of the converted terminal charts. This drastically speeds up andimproves the accuracy of having each terminal chart be reviewedmanually. In another example, the system 300 may be used to validateterminal charts after they have been ported or transmitted to anothercomputer. For example, a plurality of terminal charts may be transmittedfrom a ground based computer to a computer onboard an aircraft. Thesystem 300 is then used to confirm that all of the terminal chartstransferred over correctly. This allows the pilots or navigators of theaircraft to validate that none of the terminal charts have issues fromthe transfer between computers or over computer networks. Otherwise, thepilots or navigators would have to manually inspect each terminal chart,which would require significant amounts of pilot time to ensure theaccuracy of the terminal charts.

FIG. 4 is a simplified block diagram of an example system 400 forexecuting the systems 100 and 300 (shown in FIGS. 1 and 3). In theexample embodiment, the system 400 is used for analyzing image data todetect features in the images to identify an image category for theimage. In some embodiments, the system 400 is also used to analyze theimages for one or more errors. In addition, the system 400 is areal-time image analyzing and classifying computer system that includesan image classification and learning (ICL) computer device 410 (alsoknown as an ICL server) configured to analyze and categorize images.

As described below in more detail, the ICL server 410 is programmed toanalyze images to identify classifications for them. In addition, theICL server 410 is programmed to train the ICL model 125 (shown in FIG.1). In some embodiments, the ICL server 410 is programmed to execute oneor more of first model training module 105, second model training module130 (both shown in FIG. 1), image production 305, and data validationmodule 315 (both shown in FIG. 3). The ICL server 410 is programmed toa) execute a model for categorizing images into an image category of aplurality of image categories; b) receive an image, wherein the imageincludes an image category of the plurality of image categories; c)identify an image category for the image based on the model; d) comparethe identified image category to the included image category todetermine if there is a match; e) if there is a match, approve theimage; and f) if there is not a match, flag the image for furtherreview.

In the example embodiment, client systems 405 are computers that includea web browser or a software application, which enables client systems405 to communicate with ICL server 410 using the Internet, a local areanetwork (LAN), or a wide area network (WAN). In some embodiments, theclient systems 405 are communicatively coupled to the Internet throughmany interfaces including, but not limited to, at least one of anetwork, such as the Internet, a LAN, a WAN, or an integrated servicesdigital network (ISDN), a dial-up-connection, a digital subscriber line(DSL), a cellular phone connection, a satellite connection, and a cablemodem. Client systems 405 can be any device capable of accessing anetwork, such as the Internet, including, but not limited to, a desktopcomputer, a laptop computer, a personal digital assistant (PDA), acellular phone, a smartphone, a tablet, a phablet, or other web-basedconnectable equipment. In at least one embodiment, one or more clientsystems 405 are associated with the image production 305.

A database server 415 is communicatively coupled to a database 420 thatstores data. In one embodiment, the database 420 is a database thatincludes one or more of the first training set 120, the second trainingset 130, ICL model 125, trained ICL model 145 (all shown in FIG. 1), andthe set of images 310 (shown in FIG. 3). In some embodiments, thedatabase 420 is stored remotely from the ICL server 410. In someembodiments, the database 420 is decentralized. In the exampleembodiment, a person can access the database 420 via the client systems405 by logging onto ICL server 410.

FIG. 5 illustrates an example configuration of client system 405 (shownin FIG. 4), in accordance with one embodiment of the present disclosure.User computer device 502 is operated by a user 501. The user computerdevice 502 may include, but is not limited to, the image production 305(shown in FIG. 1) and the client systems 405 (shown in FIG. 4). The usercomputer device 502 includes a processor 505 for executing instructions.In some embodiments, executable instructions are stored in a memory area510. The processor 505 may include one or more processing units (e.g.,in a multi-core configuration). The memory area 510 is any deviceallowing information such as executable instructions and/or transactiondata to be stored and retrieved. The memory area 510 may include one ormore computer-readable media.

The user computer device 502 also includes at least one media outputcomponent 515 for presenting information to the user 501. The mediaoutput component 515 is any component capable of conveying informationto the user 501. In some embodiments, the media output component 515includes an output adapter (not shown) such as a video adapter and/or anaudio adapter. An output adapter is operatively coupled to the processor505 and operatively coupleable to an output device such as a displaydevice (e.g., a cathode ray tube (CRT), liquid crystal display (LCD),light emitting diode (LED) display, or “electronic ink” display) or anaudio output device (e.g., a speaker or headphones). In someembodiments, the media output component 515 is configured to present agraphical user interface (e.g., a web browser and/or a clientapplication) to the user 501. A graphical user interface may include,for example, an interface for viewing the image category associated withone or more images. In some embodiments, the user computer device 502includes an input device 520 for receiving input from the user 501. Theuser 501 may use the input device 520 to, without limitation, select animage to view the analysis of. The input device 520 may include, forexample, a keyboard, a pointing device, a mouse, a stylus, a touchsensitive panel (e.g., a touch pad or a touch screen), a gyroscope, anaccelerometer, a position detector, a biometric input device, and/or anaudio input device. A single component such as a touch screen mayfunction as both an output device of the media output component 515 andthe input device 520.

The user computer device 502 may also include a communication interface525, communicatively coupled to a remote device such as the ICL server410 (shown in FIG. 4). The communication interface 525 may include, forexample, a wired or wireless network adapter and/or a wireless datatransceiver for use with a mobile telecommunications network.

Stored in the memory area 510 are, for example, computer-readableinstructions for providing a user interface to the user 501 via themedia output component 515 and, optionally, receiving and processinginput from the input device 520. A user interface may include, amongother possibilities, a web browser and/or a client application. Webbrowsers enable users, such as the user 501, to display and interactwith media and other information typically embedded on a web page or awebsite from the ICL server 410. A client application allows the user501 to interact with, for example, the ICL server 410. For example,instructions may be stored by a cloud service, and the output of theexecution of the instructions sent to the media output component 515.

The processor 505 executes computer-executable instructions forimplementing aspects of the disclosure. In some embodiments, theprocessor 505 is transformed into a special purpose microprocessor byexecuting computer-executable instructions or by otherwise beingprogrammed.

FIG. 6 illustrates an example configuration of the server system 410(shown in FIG. 4), in accordance with one embodiment of the presentdisclosure. Server computer device 601 may include, but is not limitedto, the system 100 (shown in FIG. 1), the system 300, the imageproduction 305 (both shown in FIG. 3), the database server 415, and theICL server 410 (both shown in FIG. 4). The server computer device 601also includes a processor 605 for executing instructions. Instructionsmay be stored in a memory area 610. The processor 605 may include one ormore processing units (e.g., in a multi-core configuration).

The processor 605 is operatively coupled to a communication interface615 such that the server computer device 601 is capable of communicatingwith a remote device such as another server computer device 601, anotherICL server 410, or the client system 405 (shown in FIG. 4). For example,the communication interface 615 may receive requests from the clientsystem 405 via the Internet, as illustrated in FIG. 4.

The processor 605 may also be operatively coupled to a storage device634. The storage device 634 is any computer-operated hardware suitablefor storing and/or retrieving data, such as, but not limited to, dataassociated with the database 420 (shown in FIG. 4). In some embodiments,the storage device 634 is integrated in the server computer device 601.For example, the server computer device 601 may include one or more harddisk drives as the storage device 634. In other embodiments, the storagedevice 634 is external to the server computer device 601 and may beaccessed by a plurality of server computer devices 601. For example, thestorage device 634 may include a storage area network (SAN), a networkattached storage (NAS) system, and/or multiple storage units such ashard disks and/or solid state disks in a redundant array of inexpensivedisks (RAID) configuration.

In some embodiments, the processor 605 is operatively coupled to thestorage device 634 via a storage interface 620. The storage interface620 is any component capable of providing the processor 605 with accessto the storage device 634. The storage interface 620 may include, forexample, an Advanced Technology Attachment (ATA) adapter, a Serial ATA(SATA) adapter, a Small Computer System Interface (SCSI) adapter, a RAIDcontroller, a SAN adapter, a network adapter, and/or any componentproviding the processor 605 with access to the storage device 634.

The processor 605 executes computer-executable instructions forimplementing aspects of the disclosure. In some embodiments, theprocessor 605 is transformed into a special purpose microprocessor byexecuting computer-executable instructions or by otherwise beingprogrammed. For example, the processor 605 is programmed withinstructions.

At least one of the technical solutions provided by this system toaddress technical problems may include: (i) improved analysis of images;(ii) increased categorization of images; (iii) improved speed ofanalysis of images; (iv) more accurate classification; and (v) moreaccurate error analysis of images.

The computer-implemented methods discussed herein may includeadditional, less, or alternate actions, including those discussedelsewhere herein. The methods may be implemented via one or more localor remote processors, transceivers, servers, and/or sensors (such asprocessors, transceivers, servers, and/or sensors mounted on vehicles ormobile devices, or associated with smart infrastructure or remoteservers), and/or via computer-executable instructions stored onnon-transitory computer-readable media or medium.

Additionally, the computer systems discussed herein may includeadditional, less, or alternate functionality, including that discussedelsewhere herein. The computer systems discussed herein may include orbe implemented via computer-executable instructions stored onnon-transitory computer-readable media or medium.

A processor or a processing element may be trained using supervised orunsupervised machine learning, and the machine learning program mayemploy a neural network, which may be a convolutional neural network, adeep learning neural network, a reinforced or reinforcement learningmodule or program, or a combined learning module or program that learnsin two or more fields or areas of interest. Machine learning may involveidentifying and recognizing patterns in existing data in order tofacilitate making predictions for subsequent data. Models may be createdbased upon example inputs in order to make valid and reliablepredictions for novel inputs.

Additionally or alternatively, the machine learning programs may betrained by inputting sample data sets or certain data into the programs,such as images, object statistics and information, historicalcategorizations, and/or actual errors. The machine learning programs mayutilize deep learning algorithms that may be primarily focused onpattern recognition, and may be trained after processing multipleexamples. The machine learning programs may include Bayesian ProgramLearning (BPL), voice recognition and synthesis, image or objectrecognition, optical character recognition, and/or natural languageprocessing—either individually or in combination. The machine learningprograms may also include natural language processing, semanticanalysis, automatic reasoning, and/or machine learning.

Supervised and unsupervised machine learning techniques may be used. Insupervised machine learning, a processing element may be provided withexample inputs and their associated outputs, and may seek to discover ageneral rule that maps inputs to outputs, so that when subsequent novelinputs are provided the processing element may, based upon thediscovered rule, accurately predict the correct output. In unsupervisedmachine learning, the processing element may be required to find its ownstructure in unlabeled example inputs. In one embodiment, machinelearning techniques may be used to extract data about infrastructuresand users associated with a building to detect events and correlationsbetween detected events to identify trends.

Based upon these analyses, the processing element may learn how toidentify characteristics and patterns that may then be applied toanalyzing image data, model data, and/or other data. For example, theprocessing element may learn to identify the category and/or one or moreerrors in an image. The processing element may also learn how toidentify rendering errors that may not be readily apparent based uponimage data.

The methods and system described herein may be implemented usingcomputer programming or engineering techniques including computersoftware, firmware, hardware, or any combination or subset. As disclosedabove, at least one technical problem with prior systems is that thereis a need for systems for a cost-effective and reliable manner foranalyzing data to predict events. The system and methods describedherein address that technical problem. Additionally, at least one of thetechnical solutions provided by this system to overcome technicalproblems may include: (i) improved analysis of images; (ii) increasedcategorization of images; (iii) improved speed of analysis of images;(iv) more accurate classification; and (v) more accurate error analysisof images.

The methods and systems described herein may be implemented usingcomputer programming or engineering techniques including computersoftware, firmware, hardware, or any combination or subset thereof,wherein the technical effects may be achieved by performing at least oneof the following steps: (a) store a first training set of images,wherein each image of the first training set of images is associatedwith an image category of a plurality of image categories; (b) analyzeeach image of the first training set of images to determine one or morefeatures associated with each of the plurality of image categories; (c)receive a second training set of images, wherein the second training setof images includes one or more errors; (d) analyze each image of thesecond training set of images to determine one or more featuresassociated with an error category; (e) generate a model to identify eachof the image categories based on the analysis such that the modelincludes the error category in the plurality of image categories; (0receive an image, wherein the image includes an image category of theplurality of image categories; (g) execute the model to analyze thereceived image; (h) identify an image category of the plurality of imagecategories for the image based on the execution of the model; (i)compare the identified image category to the included image category todetermine if there is a match; (j) if there is a match, approve theimage; (k) if there is not a match, flag the image for further review;(1) update the model based on results of the further review; (m)identify one or more errors in the image based on a comparison of theplurality of features in the image and the one or more featuresassociated with the error category, wherein the image includes pluralityof features, wherein the error category includes a plurality ofsub-categories, wherein each sub-category identifies a different error;(n) identify the image category based on a comparison of the pluralityof features in the image and the one or more features associated withthe plurality of image categories; (o) assign the image to errorcategory when the image does not fit into any other of the plurality ofimage categories; (p) receive a plurality of images, wherein each imageincludes a metadata image category; (q) execute the model for each imageof the plurality of images; (r) associate an image category of theplurality of image categories with each image of the plurality ofimages; and (s) determine if the associated image category matches thecorresponding metadata image category.

The technical effects may also be achieved by performing at least one ofthe following steps: (a) receiving an image, wherein the image includesan image category of the plurality of image categories, wherein theimage includes plurality of features; (b) executing a model forcategorizing images into an image category of the plurality of imagecategories, wherein the model includes a plurality of features for eachimage category of the plurality of image categories; (c) identifying animage category for the image based on the model; (d) comparing theidentified image category to the included image category to determine ifthere is a match; (e) if there is a match, approving the image; (0 ifthere is not a match, flagging the image for further review; (g)identifying the image category based on a comparison of the plurality offeatures in the image and the plurality of features associated with theplurality of image categories; (h) identifying at least one error in theimage based on the comparison; (i) assigning the image to error categorywhen the image does not fit into any other of the plurality of imagecategories, wherein the plurality of image categories include an errorcategory; (j) updating the model based on results of the further review;(k) training the model with a plurality of images with errors; (1)storing a first training set of images, wherein each image of the firsttraining set of images is associated with an image category of theplurality of image categories; (m) analyzing each image of the firsttraining set of images to determine one or more features associated witheach of the plurality of image categories; (o) generating the model toidentify each of the image categories based on the analysis; (p)receiving a second training set of images, wherein a subset of thesecond training set of images includes one or more errors; and (q)updating the model with the second training set of images such that themodel includes an error category in the plurality of image categories.

The computer-implemented methods discussed herein may includeadditional, less, or alternate actions, including those discussedelsewhere herein. The methods may be implemented via one or more localor remote processors, transceivers, servers, and/or sensors (such asprocessors, transceivers, servers, and/or sensors mounted on vehicles ormobile devices, or associated with smart infrastructure or remoteservers), and/or via computer-executable instructions stored onnon-transitory computer-readable media or medium. Additionally, thecomputer systems discussed herein may include additional, less, oralternate functionality, including that discussed elsewhere herein. Thecomputer systems discussed herein may include or be implemented viacomputer-executable instructions stored on non-transitorycomputer-readable media or medium.

As used herein, the term “non-transitory computer-readable media” isintended to be representative of any tangible computer-based deviceimplemented in any method or technology for short-term and long-termstorage of information, such as, computer-readable instructions, datastructures, program modules and sub-modules, or other data in anydevice. Therefore, the methods described herein may be encoded asexecutable instructions embodied in a tangible, non-transitory, computerreadable medium, including, without limitation, a storage device and/ora memory device. Such instructions, when executed by a processor, causethe processor to perform at least a portion of the methods describedherein. Moreover, as used herein, the term “non-transitorycomputer-readable media” includes all tangible, computer-readable media,including, without limitation, non-transitory computer storage devices,including, without limitation, volatile and nonvolatile media, andremovable and non-removable media such as a firmware, physical andvirtual storage, CD-ROMs, DVDs, and any other digital source such as anetwork or the Internet, as well as yet to be developed digital means,with the sole exception being a transitory, propagating signal.

This written description uses examples to disclose variousimplementations, including the best mode, and also to enable any personskilled in the art to practice the various implementations, includingmaking and using any devices or systems and performing any incorporatedmethods. The patentable scope of the disclosure is defined by theclaims, and may include other examples that occur to those skilled inthe art. Such other examples are intended to be within the scope of theclaims if they have structural elements that do not differ from theliteral language of the claims, or if they include equivalent structuralelements with insubstantial differences from the literal language of theclaims.

What is claimed is:
 1. A system comprising: a computing devicecomprising at least one processor in communication with at least onememory device, wherein the at least one processor is programmed to:store a first training set of images, wherein each image of the firsttraining set of images is associated with an image category of aplurality of image categories; analyze each image of the first trainingset of images to determine one or more features associated with each ofthe plurality of image categories; receive a second training set ofimages, wherein the second training set of images includes one or moreerrors; analyze each image of the second training set of images todetermine one or more features associated with an error category; andgenerate a model to identify each of the image categories based on theanalysis such that the model includes the error category in theplurality of image categories.
 2. The system of claim 1, wherein the atleast one processor is further programmed to: receive an image, whereinthe image belongs to an image category of the plurality of imagecategories; execute the model to analyze the received image; andidentify an image category of the plurality of image categories for theimage based on the execution of the model.
 3. The system of claim 2,wherein the at least one processor is further programmed to: compare theidentified image category to the included image category to determine ifthere is a match; if there is a match, approve the image; and if thereis not a match, flag the image for further review.
 4. The system ofclaim 3, wherein the at least one processor is further programmed toupdate the model based on a result of the further review.
 5. The systemof claim 2, wherein the image includes plurality of features and wherethe at least one processor is further configured to identify one or moreerrors in the image based on a comparison of the plurality of featuresin the image and the one or more features associated with the errorcategory.
 6. The system of claim 5, wherein the error category includesa plurality of sub-categories, wherein each sub-category identifies adifferent error.
 7. The system of claim 2, wherein the image includesplurality of features and wherein the at least one processor is furtherprogrammed to identify the image category based on a comparison of theplurality of features in the image and the one or more featuresassociated with the plurality of image categories.
 8. The system ofclaim 7, wherein the at least one processor is further programmed toassign the image to error category when the image does not fit into anyother of the plurality of image categories.
 9. The system of claim 2,wherein the at least one processor is further programmed to: receive aplurality of images, wherein each image includes a metadata imagecategory; execute the model for each image of the plurality of images;associate an image category of the plurality of image categories witheach image of the plurality of images; and determine if the associatedimage category matches the corresponding metadata image category. 10.The system of claim 1, wherein the first training set of images and thesecond training set of images include terminal charts.
 11. A method forcategorizing images, the method implemented by a computer devicecomprising at least one processor in communication with at least onememory device, the method comprising: storing a first training set ofimages, wherein each image of the first training set of images isassociated with an image category of a plurality of image categories;analyzing each image of the first training set of images to determineone or more features associated with each of the plurality of imagecategories; receiving a second training set of images, wherein thesecond training set of images includes one or more errors; analyzingeach image of the second training set of images to determine one or morefeatures associated with an error category; and generating a model toidentify each of the image categories based on the analysis such thatthe model includes the error category in the plurality of imagecategories.
 12. The method of claim 11 further comprising: receiving animage, wherein the image belongs to an image category of the pluralityof image categories; executing the model to analyze the received image;and identifying an image category of the plurality of image categoriesfor the image based on the execution of the model.
 13. The method ofclaim 12 further comprising: comparing the identified image category tothe included image category to determine if there is a match; if thereis a match, approving the image; and if there is not a match, flaggingthe image for further review.
 14. The method of claim 13 furthercomprising updating the model based on a result of the further review.15. The method of claim 12, wherein the image includes plurality offeatures and wherein the method further comprises identifying one ormore errors in the image based on a comparison of the plurality offeatures in the image and the one or more features associated with theerror category.
 16. The method of claim 12, wherein the image includesplurality of features and wherein the method further comprisesidentifying the image category based on a comparison of the plurality offeatures in the image and the one or more features associated with theplurality of image categories.
 17. The method of claim 16 furthercomprising assigning the image to error category when the image does notfit into any other of the plurality of image categories.
 18. The methodof claim 11 further comprising: receiving a plurality of images, whereineach image includes a metadata image category; executing the model foreach image of the plurality of images; associating an image category ofthe plurality of image categories with each image of the plurality ofimages; and determining if the associated image category matches thecorresponding metadata image category.
 19. A computer-readable storagemedium comprising executable instructions that, in response toexecution, cause a system that comprises a processor, to performoperations comprising: receiving an image, wherein the image comprises ametadata image category; executing a model to analyze the receivedimage, wherein the model has been trained by analyzing a first trainingset of images, wherein each image of the first training set of images isassociated with an image category of a plurality of image categories,wherein the analyzing the first training set of images comprisesanalyzing each image of the first training set of images to determineone or more features associated with each of the plurality of imagecategories, and wherein the model has been further trained to determineone or more features associated with an error category based onanalyzing a second training set of images that comprise one or moreerrors; identifying a first image category of the plurality of imagecategories for the image based on the execution of the model; comparingthe identified image category to the metadata image category; inresponse to determining that the identified image category and themetadata image category satisfy a defined matching criterion, approvingthe image; and in response to determining that the identified imagecategory and the metadata image category do not satisfy the definedmatching criterion, flagging the image for further review.
 20. Thecomputer-readable storage medium of claim 19, wherein the first trainingset of images and the second training set of images comprise terminalcharts.